An Analysis of Stochastic Shortest Path Problems

We consider a stochastic version of the classical shortest path problem whereby for each node of a graph, we must choose a probability distribution over the set of successor nodes so as to reach a certain destination node with minimum expected cost. The costs of transition between successive nodes can be positive as well as negative. We prove natural generalizations of the standard results for the deterministic shortest path problem, and we extend the corresponding theory for undiscounted finite state Markovian decision problems by removing the usual restriction that costs are either all nonnegative or all nonpositive. * Supported by the National Science Foundation under Grant NSF-ECS-8519058, the Army Research Office under Grant DAAL03-86-K-0171, and a Presidential Young Investigator Award to the second author with matching funds from IBM, Inc. and Dupont, Inc. ** Massachusetts Institute of Technology, Laboratory for Information and Decision Systems, Cambridge, Mass. 02139. 1